A Multi-Modal Deep Learning Framework for Pixel-Level Fault Detection and Hardware Component Isolation in Satellite Imagery

Abstract

Satellites are increasingly mission-critical infrastructure, yet the tools used to monitor their health have not kept pace with the complexity of modern imaging sensors. Most operational monitoring systems still depend on telemetry thresholds a method that works well for obvious failures but consistently misses the slow, subtle degradation that begins at the pixel level. This paper describes SmartSAT Guardian, a system we built to address exactly that gap. Rather than waiting for a telemetry channel to cross a threshold, SmartSAT Guardian analyses the images a satellite captures and looks for pixel-level signatures that indicate specific hardware problems. When it finds one, it identifies not just the fault type but the physical component responsible — the photodetector array, the readout circuit, the lens mechanism, or the data downlink module. The system combines a CNN Autoencoder for anomaly detection, an ANN for fault classification, an LSTM for tracking how faults evolve over time, Granger Causality for confirming root cause through telemetry, and an NLP module that turns all of this into a readable report for ground operators. On our synthetic test dataset with five fault categories, the ANN classifier reached 94.1% accuracy with an F1-score of 92.5%. The entire pipeline runs as a decentralised web application on the Internet Computer Protocol, with no dependency on traditional cloud hosting.

Introduction

Satellites are difficult to repair once deployed in orbit, making early fault detection critical. Traditional satellite health monitoring relies on telemetry data such as temperature, battery voltage, signal strength, and altitude, with alerts triggered when values exceed predefined thresholds. While effective for detecting major failures, these methods often miss gradual hardware degradation that first appears in image quality rather than telemetry readings. Examples include dead pixels caused by failing photodetectors and image noise resulting from transmission errors. These issues may become visible long before telemetry data indicates a problem.

To address this gap, the proposed SmartSAT Guardian system uses satellite imagery as a primary diagnostic source for early fault detection. The system analyzes image anomalies, identifies the responsible hardware component, tracks fault progression, confirms fault propagation through telemetry data, and generates actionable reports for operators. It consists of five integrated modules: a CNN Autoencoder for anomaly detection, an ANN classifier for fault identification and hardware mapping, an LSTM model for temporal degradation tracking, a Granger Causality module for confirming fault propagation paths, and an NLP module for generating plain-language reports. The entire system is deployed on the Internet Computer Protocol (ICP), ensuring decentralized, reliable, and tamper-resistant operation.

The literature review examines four major research areas: telemetry-based anomaly detection, CNN-based image analysis, satellite Fault Detection, Isolation, and Recovery (FDIR) systems, and causal inference methods. Existing studies have successfully applied LSTM models to telemetry analysis, CNNs to satellite image processing, machine learning to fault isolation, and Granger Causality to disturbance propagation analysis. However, none integrate image-based fault detection, hardware attribution, temporal tracking, causal analysis, natural-language reporting, and decentralized deployment into a single framework.

The system architecture is guided by four principles: prioritizing image data over telemetry for fault detection, identifying the specific hardware component responsible for faults, monitoring fault progression over time, and ensuring decentralized deployment. The processing pipeline begins with normalized satellite images and telemetry inputs. A CNN Autoencoder generates anomaly heatmaps and fault masks, followed by an ANN classifier that identifies fault types and associated hardware components. An LSTM model tracks degradation trends and estimates time-to-failure. Granger Causality analysis validates fault propagation through telemetry channels, while an NLP module produces understandable fault reports. Results are presented to mission operators through an ICP-based dashboard.

Conclusion

SmartSAT Guardian started from a straightforward observation: the tools ground operators use to monitor satellite health were not looking at the right data. Telemetry thresholds catch failures after they have already happened. Pixel-level imaging analysis can catch them while they are still developing — and can tell you which hardware component is responsible. The system we built around that idea performs well on synthetic data. The CNN autoencoder reaches 92.5% F1-score for anomaly detection. The ANN classifier hits 94.1% accuracy across five fault categories. The LSTM tracks degradation trends with 89.3% accuracy. Every hypothesised causal link in the telemetry data was confirmed. All of it runs on a decentralised ICP deployment that does not depend on any cloud provider staying up.

References

[1] V. Chandola, A. Banerjee, and V. Kumar, \"Anomaly detec-tion: A survey,\" ACM Computing Surveys, vol. 41, no. 3, pp. 1–58, 2009. [2] K. Hundman, V. Constantinou, C. Laporte, I. Colwell, and T. Soderstrom, \"Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding,\" in Proc. 24th ACM SIGKDD Int. Conf. Knowledge Discovery & Data Mining, 2018, pp. 387–395. [3] P. Malhotra, L. Vig, G. Shroff, and P. Agarwal, \"LSTM-based encoder-decoder for multi-sensor anomaly detection,\" in Proc. ICML Anomaly Detection Workshop, 2016. [4] Y. Chen, H. Jiang, C. Li, X. Jia, and P. Ghamisi, \"Deep feature extraction and classification of hyperspectral images based on convolutional neural networks,\" IEEE Transactions on Geo-Science and Remote Sensing, vol. 54, no. 10, pp. 6232–6251, 2016. [5] X. X. Zhu et al., \"Deep learning in remote sensing: A comprehensive review,\" IEEE Geoscience and Remote Sensing Magazine, vol. 5, no. 4, pp. 8–36, 2017. [6] S. Fuertes et al., \"Improving satellite FDIR using machine learning,\" IFAC-PapersOnLine, vol. 49, no. 20, pp. 558–563, 2016. [7] M. Bauer et al., \"Finding the direction of disturbance propagation in a chemical process using transfer entropy,\" IEEE Transactions on Control Systems Technology, vol. 15, no. 1, pp. 12–21, 2007. [8] DFINITY Foundation, Motoko Programming Language Guide, 2024. [Online]. Available: https://internetcomputer.org, accessed Jan. 2026. [9] U.S. Geological Survey, \"EO-1 Hyperion Hyperspectral Imager,\" USGS Earth Resources Observation and Science Centre, 2017. [Online]. Available: https://www.usgs.gov [10] European Space Agency, \"Sentinel-2 User Handbook,\" ESA Standard Document, 2015. [Online]. Available: https://sen-tinel.esa.int [11] R. Schowengerdt, Remote Sensing: Models and Methods for Image Processing, 3rd ed. Burlington, MA: Academic Press, 2007. [12] M. Abadi et al., \"TensorFlow: A system for large-scale machine learning,\" in Proc. 12th USENIX OSDI, 2016, pp. 265–283. [13] S. S. Haykin, Neural Networks and Learning Machines, 3rd ed. New York: Pearson, 2009. [14] J. An and S. Cho, \"Variational Autoencoder-based Anomaly Detection using Reconstruction Probability,\" in Proc. Special Lecture on IE, vol. 2, no. 1, 2015.

Copyright

Copyright © 2026 Pasupuleti Koushik Kumar, Pasupuleti Naga Lakshmi Priyanka, Beemanapalli Rama Chandra , Chenna Satyannarayana , Banavath Venkata Ram . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.